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ABSTRACT 


Since the number of Mild Cognitive Impairment (MCI) older adults 
is increasing, it becomes more and more important to provide them 
with support to avoid the progression of their cognitive decline to 
dementia. To this regard, interactive serious games can play an 
important role. However, while most of them have been deployed 
mainly through tablets, the current emerging humanoid robots are 
opening up novel possibilities to this regard. In this position paper 
we aim to describe our current research interest in better 
understanding the impact of humanoid robots in supporting serious 
games for such users. 
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1 Introduction 


With a senior population that is foreseen to more than double by 
2050 worldwide [1], an increasing demand for high-quality elderly 
support is expected in the coming years. Among the disabilities 
typically associated with ageing, cognitive impairments are those 
affecting a significant part of people aged 65 plus [2]. In particular, 
Mild Cognitive Impairment (MCI) is an intermediate stage between 
the cognitive decline associated with normal ageing and more 
serious forms of dementia. Seniors with MCI often show memory 
loss or forgetfulness and may have issues with other cognitive 
functions such as language, attention and visuospatial abilities. The 
potential evolution of this disease makes it necessary to provide 
such people with increasing assistance over time. However, 
previous work indicates that cognitive training could benefit 
cognition in patients with MCI [9, 10]. Thus, it is especially 
important to offer them timely and engaging cognitive training to 
slow the progression of their decline, while significantly cutting 
down the associated socio-economic costs. 


While the introduction of computerized serious games for cognitive 
support has received several proposals [3, 4, 5], the possibilities 
that emerging interactive technologies may offer to such people 
seem largely untapped until now, also because for seniors with 
MCI, technology not only can provide opportunities, but may also 
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present potential challenges because of their difficulties in e.g. 
visual attention and memory. t Thus, to allow older adults with 
cognitive impairments to interact comfortably with technology and 
actually benefit from it, it is important to understand [6] how they 
approach and perceive the cognitive interventions provided through 
various types of assistive interaction technologies. In particular, 
humanoid robots seem promising since they can support more 
engaging interactions with users, and there have already been some 
work exploring the use of robots for supporting cognitive 
interventions [7, 8, 11]. Currently, we are focusing on humanoid 
robots and tablets to investigate how seniors with MCI relate with 
and perceive serious games accessed through humanoid robots, as 
part of a training program aimed to improve their cognitive status. 
To this goal, we designed and developed a quiz game to help the 
subjects to train their memory while receiving encouraging 
feedback from the application. We introduced not only the version 
of the game for humanoid robot but also a version for tablets, whose 
purpose was to represent a useful reference point for our analysis, 
given that the tablet is currently the most used device in cognitive 
training. 


2 The Game 


After some discussion with psychologist working on assisting and 
stimulating MCI older adults, it was decided to design an 
application requiring users to recognize songs from the years when 
they were young. Users can listen to a small segment of a song, and 
they should guess the title or the singer. After the question is 
displayed and vocally synthesized by the application, a musical 
piece is played from the beginning, and users are asked to recognize 
the title or the singer. They have a fixed time limit to respond, then 
the correct answer is shown. If they indicate a wrong answer, the 
right one is highlighted to them. The interaction was designed to 
take place in a multimodal and adaptive manner, i.e. it provides 
different feedback depending on whether the user answers correctly 
or not. Indeed, on the one hand, if the answer is wrong, the game 
highlights the provided (wrong) answer in red and displays the 
thumbs down icon in order to reinforce the negative feedback. It 
also shows the right answer (to allow users to learn it), with a green 
background and a thumbs up icon. On the other hand, if the user 


provides the right answer, a reinforcement sentence is shown (i.e. 
“very good, you gave the right answer”), and the singer and the 
song title are shown with a green background and thumbs up icon. 
While all the above-mentioned sentences are also vocally 
synthesized in both the considered cases (Pepper robot and tablet), 
one difference between the two devices is the additional feedback 
that is provided by the robot after a user provides the answer to a 
specific question. Indeed, the humanoid robot adapts its feedback 
to the older adults’ inputs in terms of additional modalities such as 
head, body, hand and arm movements, as well as sounds and 
coloured leds positioned on shoulders and eyes (which become 
green in case of right answers, red for mistakes, and blue before 
answers). In particular, in case of a question correctly answered the 
(positive) feedback provided by Pepper includes e.g. nodding its 
head, or rising its arm with a closed fist while rendering vocally a 
“hey!” expression. Either in case of a question incorrectly answered 
or a timed-out question, the feedback provided by Pepper is adapted 
accordingly, showing different animations that better simulate 
Pepper’s empathy in communicating a non-brilliant outcome (e.g. 
among others, Pepper shakes the head). Finally, when the number 
of correct answers exceeds a given threshold of the total number of 
questions, Pepper uses other gestures to render a more ‘celebrating’ 
attitude towards the user. 


Figure 1: An older adult interacting with the robot game. 


Such animations were identified so as to make the robot provide 
expressions that to some extent resemble human-like ones, so that 
users interpret the robot’s behavior as empathic as possible, 
depending on the user’s current game result. We chose not to use 
negative animations in this context, to encourage the elderly to do 
increasingly better. Users can end the game at any time. At the end 
of a session a feedback message is displayed and synthesized 
depending on the percentage of correct answers. 


3 User Feedback 


A study was carried out to assess the effectiveness of the developed 
game in improving users’ cognitive (e.g. memory-related) skills. 


Such experimental test was included in the routine training session 
offered by the clinic. In this training program, the use of personal 
computers was proposed beforehand, but it was rather 
unsuccessful. Older adults found it difficult to use and boring, so in 
the end they did not find interesting and decided to abandon it. The 
proposal of using a humanoid robot was very well received and the 
older adults expressed strong interest in trying to interact with it. 
All of the study’s participants had a diagnosis of Mild Cognitive 
Impairment, ageing over 65, and Italian speaking. They were 
invited to provide their informed written consent before the 
beginning of the study. For each participant, demographic, and 
computer experience data were formally collected in a 
questionnaire before the first session. 


The experiment was structured in two different steps: the first one 
included a user’s familiarization with the humanoid robot Pepper 
and the tablet, while in the second one users had to play the music 
game with the humanoid robot and the tablet. In this second phase, 
the users were divided into two groups, balanced according to their 
computer experience by the psychologists of the clinics. One group 
of seven participants was assigned to the group interacting with the 
Pepper robot, while the remaining seven were assigned to the 
‘tablet’ group. Overall the test results were encouraging: all the 
users successfully completed the training session until the end, and 
they were enthusiastic about participating. The game was well 
appreciated by all users, which resulted in a very competitive 
atmosphere (e.g.. participants often compared their results after the 
training). The participants did not encounter particular problems 
with understanding the vocal feedback provided by the robot. The 
caregivers also expressed satisfaction about how this experimental 
training went, and the level of users’ engagement. 


4 Conclusions and Future Plans 


In the context of interventions for reducing cognitive decline in the 
elderly population, technologies have been increasingly conceived 
as a support for patients, their caregivers and the clinicians. In this 
position paper we introduce how we are approaching the 
investigation on how seniors with Mild Cognitive Impairment 
relate with and perceive serious games accessed through humanoid 
robots, as part of a training program aimed to improve their 
cognitive status. At the workshop we also plan to report on the data 
collected in the study and further discuss their implications. 


In a next version of the game we plan to introduce different levels 
of game difficulty (e.g. asking the user to provide the title and the 
singer of a song); then the system will record the subject’s 
performance during the game and based on these data adapts the 
level of difficulty of the game and/or suggests a new game for the 
user, which will be mainly focused on the problems revealed by the 
current game. In addition, by using a reinforcement learning 
approach the system will also adapt the kind of feedback provided 
to the user after an answer, by choosing the most suitable one 
among various types of enthusiasm/sorrow expressions. 
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